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REMARKS 



Applicants have read and considered the Office Action dated December 20, 201 1, and the 
references cited therein. Claims 1 and 30 have been amended. Claims 1-3, 6-13, 15-25, 28-31, 
33, 34, 36-38 and 40-42 are currently pending. Reconsideration is hereby requested. 

Claims 1, 3, 9-10, 12-13, 15, 17-20, 23-25, 28-31, 37-38 and 41-42 were rejected under 
35 U.S.C. § 103(a) as being unpatentable over Takahashi in view of Suzuki. Applicants 
respectfully traverse the rejection. 

As discussed in the Examiner Interview on May 16, 2012, the present invention is 
fundamentally different than the cited references. Moreover, claims 1 and 30 have been 
amended and recite features that are neither shown nor suggested by either of the references or a 
combination thereof. Applicants note that support for the amendments can be found at least in 
Figures 26 and 27 as well as paragraphs 197, 199 and 208 of the published application. 

The present invention is directed to a system and method for producing stereoscopic 
images while compensating for movement of the image sensor assembly (e.g., as a result of 
human operator hand movements). An example of the system of the present invention is 
illustrated below Figure 1 : 



VISIONSENSE CLAIMED INVENTION 
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Figure 1 
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The system of the present invention acquires multiple alternating stereoscopic images by 
an image sensor. In the example set forth in Figure 1, the system acquires a first image h and a 
second image I 2 , by a sensor of 5x7 pixels. Thus, the height of Ii is given by: 8-3=5, and the 
width of i! is given by 9-2=7, the height of I 2 is given by: 7-2=5, and the width of I 2 is given by 
10-3=7. 

The system of the present invention predetermines an area size for the display images (i.e., 
output images), which is a constant size for all of the display images. In the example set forth in 
Figure 1, the predetermined and constant area size of the display images is 3x5. The system of 
the claimed invention extracts from each of the acquired images Ii and I 2 , a sub-matrix which is 
the output image. The system of the present invention therefore employs image data from only a 
portion of the pixels of the sensor (e.g., 3x5 out of 5x7) for producing an output image. In 
particular, the present system extracts Ii- ou tput from the first image Ii as the first output image, and 
extracts I 2 -out P ut from the second image I 2 as the second output image. The position of the sub- 
matrix within the respective acquired image is determined according to relative movements of 
the sensor assembly between the images. Each of the sub-matrices is imaging (i.e., showing) the 
same area of the object, thereby compensating for movements of the sensor. 

As the display images are sub-matrices of the acquired images, inevitably, the size of the 
display images is smaller than that of the acquired images. (See for example Figures 26-27) That 
is, a portion of the image sensor is "sacrificed" by omitting the image data detected thereby from 
the display images. Thereby, the display images of the present invention all have the same 
predetermined constant size, and have the same image quality as the acquired images (i.e., the 
same resolution). 

Arguments 

The monoscopic image stabilizing systems of the prior art acquire an original image and 
determine which portion of the image area is in common with a previous image (i.e., common 
image area). Such an image stabilizing system produces a new display image having the same 



12 



U.S. Patent Application Serial No. 09/785,791 
Reply to Office Action of December 20, 201 1 

area as the originally acquired image from the common image area. Those systems produce the 
new display image either by "stretching" the determined common image area thereby 
deteriorating the quality of the new image with respect to the image quality of the acquired 
image, or by supplementing the image data of the common area with anachronistic image data 
from a previously acquired image. The operation of prior art image stabilizing systems is 
detailed further below with reference to Suzuki. 

US Patent No. 5.522.789 (Takahashi) and US Patent No. 5.796.427 (Suzuki) 

Takahashi describes a stereoscopic imaging apparatus. The system of Takahashi includes 
a pair of relay lens systems, an imagery optical system and a pair of imaging devices. The 
imagery optical system is a varifocal optical system. The system detects movements of elements 
within the varifocal optical imagery system and accordingly moves the image detectors. The 
present invention is directed to producing stereoscopic images while compensating for 
movements of the sensor assembly, which are perpendicular to the optical axis of the sensor 
assembly. Takahashi fails to discuss compensating for movements of the sensor assembly as a 
result of, for example, the movement and/or shaking of the hand of the operator. This is stated in 
section 5 (page 3) of the Office Action: 

"Takahashi fails to disclose a movement detector for detecting movements of the 
sensor assembly perpendicular to the optical axis, relative to the object, and 
compensating for detected movements of the stereoscopic sensor assembly ..." 

Therefore, Suzuki is cited to address the failures of Takahashi. 

Suzuki describes a monoscopic image stabilizer as is known in the art. The operation of 
the image stabilizing method of the prior art, such as described by Suzuki, is illustrated herein in 
Figure 2: 
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SUZUKI -PRIOR ART 

^Jl-OUTPUT 




Figure 2 

The prior art system acquires a consecutive pair of images Ii and I2, each having the same 
area size 5x7. In the example set forth in Figure 2, the height of Ii is given by: 8-3=5, and the 
width of Ii is given by 9-2=7. The height of I 2 is given by: 7-2=5, and the width of I 2 is given by 
10-3=7. The prior art image stabilizer determines a common area ^-common between the first 
image Ii and the second image I 2 . In other words, common image area l2-common is the overlap 
between the first image Ii and the second image I2. Therefore, the area size of k-common is derived 
from the movement of the image sensor between acquiring the first image and acquiring the 
second image. That is, the greater the movement of the image sensor between images, the 
smaller the common area. In the example set forth in Figure 2, the area size of ^-common is 4x6. 

The prior art image stabilizer outputs the first image Ii as a first output image Ii- 0U t P ut- The 
prior art image stabilizer produces a second output image I 2 . 0 utputby "stretching" k-common to be of 
the same area size as the originally acquired images. In the example set forth in Figure 2, the 
prior art image stabilizer produces the second output image ^-output having an area size of 5x7 
from l2-common having an area size of 4x6, thereby deteriorating the quality of the second output 
image I 2 - ou tput with respect to the second image I 2 . This is a fundamentally different system and 
method than the present invention. 
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Alternatively, as presented herein in Figure 3, the common image area may be smaller: 



SUZUKI -PRIOR ART -2 




Figure 3 

Suzuki states this as follows: 

" When the common area is identified, the memory controller reads only an image 
information representing the common area . In this case, the memory controller is set 
to have, for example, a long reading period so that a third image may be built by 
means of only this image information (a less amount of information than usual) . 
After that, the image information is converted into a recording signal by the D/A 
converter. 

In the above-mentioned existing image stabilizer, there has been a problem in the 
amount of image information for representing an image to be recorded, although 
compensation of a fluctuation is performed, in case that a fluctuation of an image 
pickup device, namely, a fluctuation of an image happens. In other words, there has 
been a problem that quality of an image represented by a recording signal is 
deteriorated ." (Emphasis added) (See Suzuki, Column 2, lines 17-32) 
Suzuki aims to address the image quality deterioration of the prior art image stabilizer by 
supplementing the image data of the common area of the newly acquired image with 
anachronistic image data of the previously acquired image. The operation of the image 
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stabilizing method of Suzuki is illustrated herein in Figure 4: 



SUZUKI - SPEC. 
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Figure 4 

The Suzuki system also acquires a consecutive pair of images Ii and I 2 , each having the 
same area size 5x7. In the example set forth in Figure 4, the height of Ii is given by: 8-3=5, and 
the width of Ii is given by 9-2=7. The height of I 2 is given by: 7-2=5, and the width of I 2 is given 
by 1 1-4=7. The Suzuki image stabilizer determines a common area I 2 -common between the first 
image Ii and the second image I 2 . Therefore, common image area I 2 - CO mmon is the overlap 
between the first image Ii and the second image I 2 . In the example set forth in Figure 4, the area 
size of I 2 -common is 4x5. The common image area in Figure 4 is smaller than that of Figure 2, 
which means that the movement of the image sensor in the example set forth in Figure 2 is 
smaller than the movement in the example set forth in Figure 4. 

The Suzuki image stabilizer outputs the first image Ii as a first output image Ii. ou tput- The 
Suzuki image stabilizer produces the second output image I 2 - ou tputby supplementing I 2 . C ommon with 
the portion Ii_ part of the first image h which is not overlapped by the second image I 2 . Both the 
first output image Ii- ou tput and the second output image I 2 -out P ut have the same area size as the 
original images. The image data li. pan extracted from the first image Ii is therefore outputted as a 
portion of the second output image I 2 . 0 utput. Thereby, the second output image I 2 - 0U t P ut includes 
anachronistic image data. The Summary of Suzuki states: 
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"Namely, in case that a fluctuation of the image pickup device has been detected, an 
newly written image information corresponding to the common area and an already 
stored image information excluding this common area come to be stored in the 
second memory. In other words, an imase information, representing an image 
represented by combination of the latest image taken bv the image pickup device and 
a past image taken bv the image pickup device is read from the se cond memory:' 
(Emphasis added) 

Suzuki specifically describes supplementing the selected common area with additional 
image data from a past image such that the produced display image would have the same amount 
of image data (i.e., the same size) as the acquired images. 

"An apparatus for correcting image fluctuation in accordance with the invention can 
prevent occurrence of a problem that compensation of a fluctuation of an image 
causes reduction of an image information and can always give the same amount of 
image information as an image information produced on the basis of an image signal 
output from an image pickup device . Namely, the compensation is not accompanied 
by deterioration of quality of an image" (Emphasis added) (See Suzuki, Col.5, lines 
14-20) 

Thus, both the prior art image stabilizer and the adapted image stabilizer of Suzuki produce 
display images that are the same size as the acquired images. The display images of the prior art 
image stabilizer have a deteriorated image quality and the display images of Suzuki include 
anachronistic image data. Conversely, the present invention on the other hand, produces display 
images that are smaller than the acquired images, but which have the same image quality and 
which do not include anachronistic data. 

"wherein each one of said display images has a predetermined and constant area 

size, the area size being smaller than that of each of said detected stereoscopic 

images" (See claim 1) 

Additionally, the Office Action contends that the determined common image area of the 
image stabilizing systems described by Suzuki discloses the sub-matrices of the present 
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invention. Applicants respectfully disagree. As discussed above, the present invention 
predetermines a constant size for each sub-matrix extracted from each acquired image. The sub- 
matrices are employed as display images "as is", without any modifications, and therefore have 
to be of the same size. Conversely, the prior art image stabilizing systems determine a common 
image area for a pair of consecutive images only after the images are acquired, and not in 
advance. The common image area of the prior art is not, and cannot be, predetermined. 
Furthermore, common image area between different pairs of images can differ in terms of size 
and does not have to be constant, as exemplified in the different common image area in Figures 
2, 3 and 4. Therefore, the image stabilizing systems of the prior art do not predetermine a 
constant size for the sub-matrices. 

A combined system of Takahashi and Suzuki would include the stereoscopic endoscope 
of Takahashi complemented by the image stabilizer system of Suzuki. The combined system 
would compensate for a movement of the image detector by identifying a common image area 
between consecutive images and producing from that common image area a display image. The 
display images of the combined system of Takahashi and Suzuki would have the area size of the 
acquired images and therefore would either have lower image quality or include anachronistic 
image data. Thus, as mentioned above, the combined system compensates for movements of the 
image detector in a different manner than that of the present invention, and accordingly produces 
different display images. 

Additionally, the image stabilizing system of Suzuki operates on a single channel and 
would produce output images for the right channel and output images for the left channel 
independently. Thereby, the imaged area of the imaged object (i.e., or imaged scene) in the 
output images of the right channel would not correspond to that of the left channel. This lack of 
correspondence between the output images of the right channel and those of the left channel 
negates stereoscopy and thus, the combined system of Takahashi and Suzuki cannot produce 
stereoscopic images while compensating for movements of the sensor assembly. 
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For at least the above reasons, the combined system cannot be considered as disclosing or 
suggesting the present invention, and in particular claim 1 . The arguments presented herein 
above with respect to a system of the present invention and with respect to independent claim 1 
also apply to the method recited in independent claim 30. As independent claims 1 and 30 are 
new, novel and non-obvious over the combined teachings of Takahashi and Suzuki (and any 
other cited prior art), the dependent claims are also allowable over the prior art. 

Claims 2 and 33-34 were rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Takahashi and Suzuki in view of Adelson (US 5,076,687). Adelson fails to remedy the 
shortcomings of the Takahashi and Suzuki combination. Therefore, Applicants assert that claims 
1 and 30 patentably distinguish over the combination of Takahashi, Suzuki and Adelson. As 
claims 1 and 30 are allowable over the combination of Takahashi, Suzuki and Adelson, 
Applicants assert that claims 2 and 33-34 are allowable over the combination for at least the 
same reasons. Applicants therefore request that the rejection under 35 U.S.C. § 103(a) be 
withdrawn. 

Claim 36 was rejected under 35 U.S.C. § 103(a) as being unpatentable over Takahashi, 
Suzuki and Adelson in view of Watanabe (US 5,812,187). Watanabe fails to remedy the 
shortcomings of the Takahashi, Suzuki and Adelson combination. Therefore, Applicants assert 
that claim 30 patentably distinguishes over the combination of Takahashi, Suzuki, Adelson and 
Watanabe. As claim 30 is allowable over the combination of Takahashi, Suzuki, Adelson and 
Watanabe, Applicants assert that claim 36 is allowable over the combination for at least the same 
reasons. Applicants therefore request that the rejection under 35 U.S.C. § 103(a) be withdrawn. 

Claim 40 was rejected under 35 U.S.C. § 103(a) as being unpatentable over Takahashi and 
Suzuki in view of Watanabe. Watanabe fails to remedy the shortcomings of the Takahashi and 
Suzuki combination. Therefore, Applicants assert that claim 30 patentably distinguishes over the 
combination of Takahashi, Suzuki and Watanabe. As claim 30 is allowable over the 
combination of Takahashi, Suzuki and Watanabe, Applicants assert that claim 40 is allowable 
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over the combination for at least the same reasons. Applicants therefore request that the 
rejection under 35 U.S.C. § 103(a) be withdrawn. 

Claims 6, 8 and 1 1 were rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Takahashi and Suzuki in view of Watanabe. Watanabe fails to remedy the shortcomings of the 
Takahashi and Suzuki combination. Therefore, Applicants assert that claims 1 and 30 patentably 
distinguish over the combination of Takahashi, Suzuki and Watanabe. As claims 1 and 30 are 
allowable over the combination of Takahashi, Suzuki and Watanabe, Applicants assert that 
claims 6, 8 and 1 1 are allowable over the combination for at least the same reasons. Applicants 
therefore request that the rejection under 35 U.S.C. § 103(a) be withdrawn. 

Claim 16 and 21-22 were rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Takahashi, Suzuki and Watanabe (US 5,812,187). Watanabe fails to remedy the shortcomings of 
the Takahashi and Suzuki combination. Therefore, Applicants assert that claim 1 patentably 
distinguishes over the combination of Takahashi, Suzuki and Watanabe. As claim 1 is allowable 
over the combination of Takahashi, Suzuki and Watanabe, Applicants assert that claims 16 and 
21-22 are allowable over the combination for at least the same reasons. Applicants therefore 
request that the rejection under 35 U.S.C. § 103(a) be withdrawn. 

Conclusion 

As demonstrated above, the present invention is neither disclosed nor suggested by any of 
the cited prior art references or combination thereof. A speedy and favorable action in the form 
of a Notice of Allowance is hereby solicited. If the Examiner feels that a telephone interview 
may be helpful in this matter, please contact Applicants' representative at (612) 336-4728. 
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Please consider this a PETITION FOR EXTENSION OF TIME for a sufficient number 
of months to enter these papers or any future reply, if appropriate. Please charge any additional 
fees or credit overpayment to Deposit Account No. 13-2725. 
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Respectfully submitted, 



MERCHANT & GOULD P.C. 




GAS/mep 
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